Cross - Pose Facial Expression Recognition

¨ Ozgür for participating in my thesis jury, their criticism and valuable feedback. Engin, Kaan Bekmezci, and Safiye C ¸ elik for always being there to have fun with. I am especially grateful to my dear friend, Beyza Ermi¸s for alleviating the burden of this painful process. I would also like to thank the members of MediaLab and PILAB for providing an enjoyable working environment. I would like to express my deepest gratitude to my parents S ¸aziye and Beytul-lah Güney, and my sister Bü¸sra Güney for their endless love, support and patience throughout this process. I am specifically grateful to N. Murat Arar. I honestly could not bear the burden without his support and motivation. iv ABSTRACT CROSS-POSE FACIAL EXPRESSION RECOGNITION Automatic facial expression recognition is a popular research topic due to its interesting applications in a wide variety of areas. The existing studies have achieved high accuracies in various formulations of the same problem. One direction which is not fully explored is multi-view facial expression recognition. Variations caused by different poses impose extra burden on the task of recognizing expressions, which is already a difficult problem due to large differences across subjects. In this thesis, we present a method to recognize six prototypic facial expressions of an individual across different pose angles. We use Partial Least Squares (PLS) to map the expressions from different poses into a common subspace, in which correlation between them is maximized. Recently, PLS has been successfully used for pose invariant face recognition problem. We show that, PLS can be effectively used for facial expression recognition across poses by training on coupled expressions of the same identity from two different poses. This way of training lets the learned bases model the differences between expressions of different poses by excluding the effect of the identity. We first align the faces and then extract block features around two eyes and the mouth on the aligned image. We experiment with Gabor filters and direct intensity values for local face representation. We demonstrate that two representations perform similarly in case frontal is the input pose, but Gabor representation outperforms intensity representation for other pose pairs. We also perform a detailed analysis of the parameters used in the experiments to show their effects on the results and to find the optimal ones for the expression recognition problem.